
CLAIMS 

We Claim: 

1 . A computer implemented method for assisting a user in the cfontrol and 
operation of a computer system, the computer system having a display device, the 
5 computer system providing information content for display, sxym information content 
potentially containing more content such as characters, pictures, lines, or pixels than 
can be conveniently displayed entirely on the display device at one time, the computer 
implemented method comprising the acts of: 

coupling a display device to a computer sy^fem; 

10 mapping information content generatea by the computer system into a virtual 

desktop suitable for conveying the information to the user; 

displaying a certain portion o^flie virtual desktop using the computer system's 
display device; 

tracking movements oKhe display device; and 

15 adjusting the displayed certain portion of the virtual desktop in a manner 

related to the tracked urfovements of the display device, whereby the user is able to 
traverse the entire information content mapped to the virtual desktop and examine any 
certain portion or segment of the information content using the computer system's 
display devic^f 



20 



2. A computer implemented method as recited in claim 1 wherein a virtual 
magnification of the displayed certain portion is updated in a manner correlated to the 
tracked movement of the display device. 



25 3. A computer implemented method as recited in claim 1 wherein a virtual 

magnification of the displayed certain portion is updated in response to a command 
entered into the computer system by a user of the computer system. 
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4. A computer implemented method as recited in claim 1 further comprising the 
act of redefining the orientation of the certain portion displayed via the display device 
such that, without moving the display device, the certain portion displayed via the 
display device changes. 



5. A computer implemented method as recited in claim 4 wherein the orientation 
of the certain portion displayed is redefined in response to a request by a user. 



6. A computer implemented method as recited in claim 1 wherein a first 
10 application executing upon the computer system is a physical map application 
providing a virtual map, the movement of the display device enabling visual 
navigation through the virtual map. 



7. A computer implemented method as recited in claim 6 wherein the navigation 
15 capability of the physical map includes north, south, east, and west directional 
navigation through the virtual map. 



8. A computer implemented method as recited in claim 7 wherein the navigation 
capability of the physical map further includes a scalability feature allowing 
20 adjustment of the scalability of the physical map in order to provide a viewer of the 
display device views of the physical map having different magnifications. 



9. A computer implemented method as recited in claim 8 wherein the scalability 
feature is controlled according to tracked movement of the display device. 



10. A computer implemented method as recited in claim 8 wherein the scalability 
feature is controlled by user input separate from tracked movement of the display 
device. 
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11. A computer implemented method as recited in claim 6 wherein the navigation 
capability of the physical map includes a scalability feature allowing adjustment of 
the scalability of the physical map in order to provide a viewer of the display device 
views of the physical map having different magnifications 
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12. A computer implemented method as recited in claim 1 1 wherein the scalability 
feature is controlled according to tracked movement of the display device. 



13. A computer implemented method as recited in claim 1 1 wherein the scalability 
10 feature is controlled by user input separate from tracked movement of the display 



14. A computer implemented method as recited in claim 1 wherein the display 
device and the computer system are formed in a single computer device provided to a 
15 user of the computer device. 



15. A computer implemented method as recited in claim 14 wherein the computer 
device is a hand-held computer device. 



20 16. A computer implemented method as recited in claim 15 wherein the hand-held 
computer device is a personal digital assistant (PDA). 

17. A computer implemented method as recited in claim 16 wherein the PDA has 
handwriting recognition capability. 

25 

18. A computer implemented method as recited in claim 16 wherein the PDA has 
voice recognition capability. 
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device. 



19. A computer implemented method as recited in claim 1 wherein the visual 
information generated by the computer system includes multiple application windows. 

20. A computer implemented method as recited in claim 19 wherein a first 
window of the multiple application windows corresponds to a first application 
executing upon the computer system. 

21 . A computer implemented method as recited in claim 20 wherein the first 
application executing upon the computer system is a physical map application. 



22. A computer implemented method as recited in claim 21 wherein the physical 
map application enables navigation through a physical map via user movement of the 
display device. 



23. A computer implemented method as recited in claim 1 wherein the displayed 
certain portion or segment of the virtual desktop is adjusted in a manner related to the 
tracked movements of the display device in relation to a substantially planar surface. 



24. A computer implemented method as recited in claim 23 wherein a virtual 
magnification of the displayed certain portion is updated in response to a command 
entered into the computer system by a user of the computer system. 

25. A computer implemented method as recited in claim 24 wherein the display 
device and the computer system are formed in a single device provided to a user of 
the computer device. 

26. A computer implemented method as recited in claim 25 wherein the computer 
device is a hand held computer device. 



FATHP009/BRC/JAB 



20 



Patent 



# • 



27. A computer implemented method as recited in claim 26 wherein the hand held 
computer device is a personal digital assistant (PDA). 



28. A computer implemented method as recited in claim 25, wherein the hand 
held computer device is coupled to a second computer. 



29. A computer implemented method as recited in claim 28, further comprising 
the act of utilizing the hand held computer device to select information displayed on 
the second computer. 



30. A computer implemented method as recited in claim 22 further comprising the 
acts of: 

monitoring a real scene in real space and time; 
capturing images of the real scene; and 

displaying within a first window of the multiple application windows the 
captured images of the real scene. 



31 . A computer implemented method as recited in claim 30 wherein a second 
window of the multiple application windows corresponds to an application program 
executing upon the computer system. 



32. A method for visually navigating a virtual m#p generated by a physical map 
application executing upon a hand-held computea / system, the hand-held computer 
system having a display device and a motion/censor, the method comprising the acts 
of: / 

transforming visual information generated by the physical map application 
into a virtual map suitable for display via the display device; 
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5 correlated to t>€^Eracked movement of the hand-held computer system. 



33. A computer implemented method as recited in claim 32 further comprising the 
act of redefining the orientation of the certain portion displayed via the display device 
such that, without moving the hand-held computer system, the certain portion 

10 displayed via the display device changes. 

34. A computer implemented method as recited in claim 32 wherein the 
orientation of the certain portion displayed is redefined in response to a request by a 
user. 



35. A computer implemented method as recited in claim 32 wherein a virtual 
magnification of the displayed certain portion is updated in a manner correlated to the 
tracked movement of the hand-held computer system. 



20 36. A computer implemented method as recited in claim 32 wherein a virtual 
magnification of the displayed certain portion is updated in response to a command 
entered into the computer system by a user of the hand-held computer system. 



37. A computer implemented method as recited in claim 32 wherein the physical 
25 map application is a first application executing upon the hand-held computer system. 



38. A computer implemented method as recited in claim 32 wherein the 
navigation capability of the physical map includes north, south, east, and west 
directional navigation through the virtual map. 
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39. A computer implemented method as recited in claim 38 wherein the 
navigation capability of the physical map further includes a scalability feature 
allowing adjustment of the scalability of the physical map in order to provide a viewer 
5 of the display device views of the physical map having different magnifications. 



40. A computer implemented method as recited in claim 39 wherein the scalability 
feature is controlled according to tracked movement of the hand-held computer 
system. 

10 

41. A computer implemented method as recited in claim 39 wherein the scalability 
feature is controlled by user input separate from tracked movement of the hand-held 
computer system. 



15 42. A computer implemented method as recited in claim 32 wherein the 

navigation capability of the physical map includes a scalability feature allowing 
adjustment of the scalability of the physical map in order to provide a viewer of the 
display device views of the physical map having different magnifications 



20 43. A computer implemented method as recited in claim 42 wherein the scalability 
feature is controlled according to tracked movement of the hand-held computer 
system. 



44. A computer implemented method as recited in claim 42 wherein the scalability 
25 feature is controlled by user input separate from tracked movement of the hand-held 
computer system. 



45. A computer implemented method as recited in claim 32 wherein the hand-held 
computer system is a personal digital assistant (PDA). 
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46. A computer implemented method as recited in claim 45 wherein the PDA has 
voice recognition capability. 

47. A computer implemented method as recited in claim 32 wherein the displayed 
5 certain portion or segment of the virtual desktop is adjusted in a manner related to the 

tracked movements of the display device in relation to a substantially planar surface. 

48. A computer implemented method as recited in claim 47 wherein a virtual 
magnification of the displayed certain portion is updated in response to a command 

10 entered into the computer system by a user of the computer system. 



49. A computer implemented method as recited in claim 48 wherein the display 
device and the computer system are formed in a single device provided to a user of 
the computer device. 



50. A computer implemented method as recited in claim 49 wherein the computer 
device is a hand held computer device. 



51. A computer implemented method as recited in claim 45 wherein the PDA has 
20 handwriting recognition capability. 



52. A computer implemented method as recited in claim 50 wherein the hand held 
computer device is a personal digital assistant (PDA). 



25 53. A computer implemented method as recited in claim 49, wherein the hand 
held computer device is coupled to a second computer. 
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54. A computer implemented method as recited in claim 53, further comprising 
the act of utilizing the hand held computer device to select information displayed on 
the second computer. 




A hand-held computer system compri/ing: 
a digital processor; 

a motion sensor coupled to a disp4ay device; 

the display device coupled to the digital processor; and 

a computer readable medium coupled to the digital processor, the computer 
10 readable medium having computei/executable instructions for: 

mapping visual information generated by the computer system 
into a virtual desktop suitable for display via the display device; 

displaying a certain portion of the virtual desktop via the 
display device; 

15 tracking m^ement of the hand-held computer system via the 

motion sensor; 

updating the displayed certain portion of the virtual desktop in 
a manner correlated to the tracked movement of the hand-held 
computer syj/tem. 

20 

56. A hand held computer system as recited in claim 55 wherein the computer 
readable medium further comprises computer executable instructions for redefining 
the orientation of the certain portion displayed via the display device such that, 
without moving the display device, the certain portion displayed via the display 
25 device changes. 



57. A hand held computer system as recited in claim 56 wherein the orientation of 
the certain portion displayed is redefined in response to a request by a user. 
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58. A hand held computer system as recited in claim 55 wherein the computer 
readable medium further comprises computer executable instructions for updating a 
virtual magnification of the displayed certain portion in a manner correlated to the 
5 tracked movement of the display device. 



59. A hand held computer system as recited in claim 55 wherein the computer 
readable medium further comprises computer executable instructions for updating a 
virtual magnification of the displayed certain portion in response to a command 
10 entered into the computer system by a user of the computer system. 



60. A hand held computer system as recited in claim 55 wherein the computer 
readable medium further comprises computer executable instructions for a physical 
map application providing a virtual map, the movement of the display device enabling 

15 visual navigation through the virtual map. 

61. A hand held computer system as recited in claim 60 wherein the navigation 
capability of the physical map includes north, south, east, and west directional 
navigation through the virtual map. 



62. A hand held computer system as recited in claim 61 wherein the navigation 
capability of the physical map further includes a scalability feature allowing 
adjustment of the scalability of the physical map in order to provide a viewer of the 
display device views of the physical map having different magnifications. 



63. A hand held computer system as recited in claim 62 wherein the scalability 
feature is controlled according to tracked movement of the display device. 
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64. A hand held computer system as recited in claim 62 wherein the scalability 
feature is controlled by user input separate from tracked movement of the display 
device. 



5 65. A hand held computer system as recited in claim 60 wherein the navigation 
capability of the physical map includes a scalability feature allowing adjustment of 
the scalability of the physical map in order to provide a viewer of the display device 
views of the physical map having different magnifications 



10 66. A hand held computer system as recited in claim 65 wherein the scalability 
feature is controlled according to tracked movement of the display device. 



67. A hand held computer system as recited in claim 65 wherein the scalability 
feature is controlled by user input separate from tracked movement of the display 
15 device. 



68. A hand held computer system as recited in claim 55 wherein the hand-held 
computer system is a personal digital assistant (PDA). 



20 69. A hand held computer system as recited in claim 68 wherein the PDA has 
handwriting recognition capability. 



70. A hand held computer system as recited in claim 68 wherein the PDA has 
voice recognition capability. 

71 . A hand held computer system as recited in claim 55 wherein the visual 
information generated by the computer system includes multiple application windows. 
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72. A hand held computer system as recited in claim 71 wherein a first window of 
the multiple application windows corresponds to a first application executing upon the 
computer system. 



73. A hand held computer system as recited in claim 72 wherein the first 
application executing upon the computer system is a physical map application. 



74. A hand held computer system as recited in claim 73 wherein the physical map 
application enables navigation through a physical map via user movement of the 
display device. 



75. A hand held computer system as recited in claim 55 wherein the displayed 
certain portion of the virtual desktop is adjusted in a manner related to the tracked 
movements of the display device in relation to a substantially planar surface. 



76. A hand held computer system as recited in claim 75 wherein the computer 
readable medium further comprises computer executable instructions for updating a 
virtual magnification of the displayed certain portion is updated in response to a 
command entered into the computer system by a user of the computer system. 



77. A hand held computer system as recited in claim 76 wherein the display 
device and the computer system are formed in a single device provided to a user of 
the computer device. 



78. A hand held computer system as recited in claim 77 wherein the computer 
device is a hand held computer device. 



79. A hand held computer system as recited in claim 78 wherein the hand held 
computer device is a personal digital assistant (PDA). 
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80. A hand held computer system as recited in claim 77, wherein the hand held 
computer device is coupled to a second computer. 

81 . A hand held computer system as recited in claim 80, wherein the hand held 
computer device is utilized to select information displayed on the second computer. 

82. A hand held computer system as recited in claim 74 wherein the computer 
readable medium further comprises computer executable instructions for: 

monitoring a real scene in real space and time; 

capturing images of the real scene; and 

displaying within a first window of the multiple application windows the 
captured images of the real scene. 

83. A hand-held computer system comprising: 
a digital processor; / 

a motion sensor coupled to j^iisplay device; 

the display device couple/ to the digital processor; and 

a computer readable medium coupled to the digital processor, the computer 

readable medium having computer executable instructions for: 
a physical rriap application; 

transforming visual information generated by the physical map application 
into a virtual map suitable for display via the display device; 

displaying a certain portion of the virtual map via the display device; 

tracking movement of the hand-held computer system using the motion 
sensor; yfind 
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updating the displayed certain portion^f^fte virtual map in a manner 
correlated to the tracked movementoflliehand-held computer system. 




84. A hand held computer system as recited in claim 83 wherein the computer 
5 readable medium further comprises computer executable instructions for redefining 
the orientation of the certain portion displayed via the display device such that, 
without moving the hand-held computer system, the certain portion displayed via the 
display device changes. 



10 85. A hand held computer system as recited in claim 84 wherein the orientation of 
the certain portion displayed is redefined in response to a request by a user. 



86. A hand held computer system as recited in claim 83 wherein the computer 
readable medium further comprises computer executable instructions for updating a 
15 virtual magnification of the displayed certain portion in a manner correlated to the 
tracked movement of the hand-held computer system. 



87. A hand held computer system as recited in claim 83 wherein the computer 
readable medium further comprises computer executable instructions for updating a 
20 virtual magnification of the displayed certain portion in response to a command 
entered into the computer system by a user of the hand-held computer system. 



88. A hand held computer system as recited in claim 83 wherein the physical map 
application is a first application executing upon the hand-held computer system. 



89. A hand held computer system as recited in claim 83 wherein the computer 
readable medium further comprises computer executable instructions for navigation 
of the physical map wherein the navigation capability of the physical map includes 
north, south, east, and west directional navigation through the virtual map. 



25 
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3^^/^°- A hand held computer system as recited in cl^itn 89 wherein the navigation 
capability of the physical map further includes scalability feature allowing 
adjustment of the scalability of the physicaj/fnap in order to provide a viewer of the 
display device views of the physical majlnaving different magnifications. 



91 . A hand held computer system as recited in claim 90 wherein the scalability 
feature is controlled according to tracked movement of the hand-held computer 
system. 



10 92. A hand held computer system as recited in claim 90 wherein the scalability 
feature is controlled by user input separate from tracked movement of the hand-held 
computer system. 



93. A hand held computer system as recited in claim 83 wherein the computer 
15 readable medium further comprises computer executable instructions for navigation 

of the physical map wherein the navigation capability of the physical map includes a 
scalability feature allowing adjustment of the scalability of the physical map in order 
to provide a viewer of the display device views of the physical map having different 
magnifications 

20 

94. A hand held computer system as recited in claim 93 wherein the scalability 
feature is controlled according to tracked movement of the hand-held computer 
system. 



25 95. A hand held computer system as recited in claim 93 wherein the scalability 
feature is controlled by user input separate from tracked movement of the hand-held 
computer system. 



96. A hand held computer system as recited in claim 83 wherein the hand-held 
30 computer system is a personal digital assistant (PDA). 
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97. A hand held computer system as recited in claim 96 wherein the PDA has 
handwriting recognition capability. 



5 98. A hand held computer system as recited in claim 96 wherein the PDA has 
voice recognition capability. 



99. A hand-held computer system comprising: 
a digital processor; 

10 a motion sensor coupled to the digital processor, the motion sensor capable of 



sensing motion relative to a substantially planar surface; 



a display device coupled to the digital processor; and 



a computer readable medium coupled to the digital processor, the computer 
readable medium having computer executable instructions for: 



15 



mapping visual information generated by the computer system into a 
virtual desktop suitable for display via the display device; 



displaying a certain portion of the virtual desktop via the display 
device; 



20 



tracking movement of the hand-held computer system via the motion 
sensor; and 



updating the displayed certain portion of the virtual desktop in a 
manner correlated to the tracked movement of the hand-held computer 
system in relation to a substantially planar surface. 
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